Continual Pre-Training of Language Models for Concept Prerequisite Learning with Graph Neural Networks

Prerequisite chains are crucial to acquiring new knowledge efficiently. Many studies have been devoted automatically identifying the prerequisite relationships between concepts from educational data. Though effective some extent, these methods neglected two key factors: most works failed utilize domain-related enhance pre-trained language models, thus making textual representation of less effective; they also ignore fusion semantic information and structural formed by existing prerequisites. We propose a two-stage concept learning model (TCPL), integrate above factors. In first stage, we designed continual pre-training tasks for domain-adaptive task-specific enhancement, obtain better representation. second leverage complementary effects information, optimized encoder resource–concept graph simultaneously, with hinge loss as an auxiliary training objective. Extensive experiments conducted on three public datasets demonstrated effectiveness proposed approach. Our improved 7.9%, 6.7%, 5.6%, 8.4% ACC, F1, AP, AUC average, compared state-of-the-art methods.